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The growth of the Internet has impacted on virtually every aspect of society. Survey 
research is no exception. Two years ago in an informal search of Yahoo, Kay and 
Johnson (1999) identified over 2,000 Web-based surveys(l) in 59 areas. The interest in 
Web-based surveying is not surprising as it offers a number of distinct advantages over 
more traditional mail and phone techniques. Examples include reducing the time and 
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cost of conducting a survey and avoiding the often error prone and tedious task of data 
entry (Medin, Roy & Ann, 1999). 

Email offers one option for distributing Internet surveys. Up until a few years ago email 
surveys were the predominate means of Internet surveying. As the World Wide Web 
(WWW) has grown in popularity, the use of Hypertext Markup Language (HTML) forms 
or Web-based surveys are becoming the dominant method of gathering survey data. 
These forms streamline the data collection process formatting and entering responses 
directly into a database for analysis. Since HTML forms can be made programmable, it 
is also possible to have real time error checking and correction increasing the accuracy 
of the data collection process. The formatting capabilities of HTML allow the creation of 
easy-to-read and a attractive forms that may improve response rates. In addition, the 
programmability of HTML forms makes it possible to randomly order responses and 
tailor options based on information the respondent supplies earlier in the survey. 

Combining an email "cover letter" as a means of contacting sampled people with the 
use of an HTML form for data collection provides an especially effective and efficient 
approach to Internet surveying. Modern email packages automatically convert universal 
resource locators (URLs) or web-addresses in the text of an email into a hyperlinks. 
Placing the URL of the survey form in a cover letter email allows the respondent to 
"click" their mouse on the URL to display the survey form and subsequently fill it out. 

CONCERNS WITH WEB -BASED SURVEYING 



Although Web-based surveying is very attractive, at this point it should be used with 
caution. Currently the biggest concern in Internet surveying is coverage bias or bias due 
to sampled people not having or choosing not to access the Internet (Kay & Johnson, 
1999; Crawford, Couper & Lamias, 2001). Despite expediential growth of the Internet 
there are still large numbers of people who do not have access and/or choose not to 
use the Internet. It is also clear that there are wide disparities in Internet access among 
ethnic and socioeconomic groups (Selwyn & Robson, 1998). 

There are specific populations where Internet access is extremely high and coverage 
bias is likely to be less of a concern. College students and university faculty within the 
USA, Canada and Western Europe are examples of such populations. Even though 
coverage bias may be less of an issue in these groups, experience and comfort with 
Internet-based tools such as Web browsers is another serious potential source of bias 
both in response rates and the way people respond to the survey (Dillman, Tortora & 
Bowker, 2001). 

Web-based surveying is still in the early stages of development. The WWW is a unique 
media and it is not clear to what extent the knowledge we have gained over years of 
experience with more traditional surveying techniques fully applies to Internet surveying 
(Dilman, Tortora & Bowker, 2001). Studies are just beginning to done to learn the 
optimal ways to structure and format Internet surveys to limit biases and increase 
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response rates. It is also likely that the best way to design an Internet survey depends in 
part on the familiarity and comfort of the respondents in using Web browsers and email 
clients. It is also quite likely that the type of Internet connection as well as the hardware 
and software used in accessing the Internet will impact on response rates and possibly 
how a person responds to an Internet-based survey. 

The use of HTML forms for surveying poses a unique set of issues and challenges that 
need to be addressed to ensure valid data. The Web is a very public place and unless 
steps are taken to limit access to a survey, it may be found and responded to by people 
who are not among those sampled by the researcher. This can either happen by 
accident or maliciously. Since one only has to "click" their mouse pointer on the "submit" 
button to respond to a Web-based survey instrument once it is filled out, it is also quite 
possible for respondents to either mistakenly or purposefully submit multiple copies of 
their responses. 

While Internet-based surveying techniques need to be used with caution, their benefits 
warrant continued exploration and the cautious use. It is also pretty clear that coverage 
bias and familiarity with Internet tools will be less and less of an issue over time. 
Additionally our knowledge about how best to conduct Internet surveys will continue to 
improve with research and experience. 

RESEARCH ON INTERNET-BASED SURVEYING 



Although the research on Internet-based surveying is limited, findings are beginning to 
appear in the literature. Several studies have found that response rates for Internet 
surveys are lower that equivalent mail surveys (Medin, Roy & Ann, 1999; Cooper, Blair 
& Triplett, 1999). As noted by Crawford and colleagues (2001), this may be due to our 
lack of knowledge on how to achieve high response rates using the Internet surveys. 

The lower response rates for internet surveys may also reflect coverage bias, the lack of 
familiarity with the media and/or lack of convenient access to the Internet. In the 
author's experience, Web congestion can also be a factor in lowering response rates for 
Web surveys particularly with people who have relatively little experience with the 
Internet. Cook and colleagues (2000) conducted a meta-analysis of factors influencing 
response rates in Internet-based surveys2. They found that follow-up contacts with 
non-respondents, personalized contacts, and contacting sampled people prior to 
sending out the survey were the three dominant factors in higher response rates. 
Kittleson (1997) in a study of email-based surveying found it was possible to double the 
response rate with follow-up memos though in general this may be somewhat optimistic. 
As with mailed surveys, repeated follow-ups have diminishing returns and at some point 
risk irritating potential respondents without noticeably increasing response rates. 
Additionally, Dillman, Tortora, Conrad & Bowker (2001) found that relatively plain Web 
surveys that load quickly resulted in higher response rates than "fancier" surveys that 
take longer to load. Jeavons (1998) analyzed detailed server logs from three separate 
large-scale surveys. He found a relatively high percentage of potential respondents 
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stopped completing the surveys 1) when encountering the first question, 2) when 
encountering a complex question grid, and 3) when asked to supply their email address. 
This suggests that some potential respondents have difficulty with the media and give 
up early in the process of completing the survey or when encountering complex 
questions. Others may be reluctant to give out personal information such as an email 
address. The logs were also merged with demographic data collected via the surveys. 
Somewhat surprisingly no patterns in failure to complete rates were found by gender, 
age or education level. In two of the surveys, people with lower income were found to 
have a higher rate of repeating screens of questions mainly due to improperly filling out 
questions. 

DEVELOPING WEB SURVEYS 



As noted, due to their inherent advantages, most Internet surveying is now being done 
using HTML forms with potential respondents often contacted via email cover letters. 
While some developers still directly code these forms in HTML, there are dozens of 
HTML editors available, and they are becoming increasingly sophisticated and easy to 
use. There are two general methods of capturing the data entered into an HTML form. 
The form can be programmed to email the data back to a specified email address or 
captured by a program on the server called a common gateway interface (CGI) script. 
Using CGI scripts is more robust, offers more flexibility and is the far more commonly 
used method of capturing data. There are several HTML development packages that 
both provide HTML editing capabilities and automate the process of developing the CGI 
scripts necessary to capture data from HTML forms developed with the package. Two 
widely used examples of these packages are Microsoft's Frontpage and Macromedia's 
ColdFusion. While these packages are general purpose Web development tools, there 
are also a growing number of software development systems designed specifically for 
Web-based surveying. Examples include Perseus's Survey Solutions for the Web, 
Creative Research System's The Survey System, and Survey Said Survey Software. 
These packages tend to offer additional features specific to survey research. Examples 
include as managing the distribution of email cover letters, built-in statistical analysis 
and reporting capabilities, and automatic tracking of people who have responded 
coupled with the ability of sending out follow-up email reminders to those who have yet 
to respond. Their HTML editors are also geared for survey form development, allowing 
them to simplify and streamline the process of developing and formatting the question 
response fields. Web Survey Mailer System 

The author has developed a set of software tools that provides many of the complex 
Web survey administration functions included in Web surveying packages(3). The 
software, Web Survey Mailer System (WSMS), is an integrated survey administration 
system that will send out personalized email cover letters, track which of the sampled 
people have completed the survey while keeping their responses anonymous and send 
out subsequent reminder emails to only those sampled people who have not responded 
to the survey. WSMS will block people who have not been sampled from accessing and 
responding to the survey and will keep respondents from submitting more than one set 
of survey responses. The system includes a customizable CGI script to capture the 
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survey responses and place them in a tab-delimited ASCII database format that can 
easily be downloaded from the server and imported into a standard PC data base or 
statistical package. WSMS is written in PHP and uses the MySQL relational database to 
store information on the sampled people. Both PHP and MySQL are stable and 
powerful "open source" packages widely available on university and commercial Web 
servers and can be obtained free of charge in a variety of versions that will run on most 
common server operating systems. The WSMS scripts and documentation are available 
free-of-charge and can be downloaded from 
http://www.med-ed- 0 nline. 0 rg/rs 0 ftware.htm#wsms 

SUMMARY 



Internet surveys are clearly going to continue to grow in popularity as the problems of 
coverage bias and unfamiliarity with the Internet subside. For the foreseeable future 
there will be people who will lack Internet access either by choice or circumstance 
though this will be less and less of an issue. Additionally the tools for conducting 
Web-based surveys will continue to grow in sophistication and ease of use as will our 
knowledge on how best to employ this survey methodology. At present researchers 
should use this technique with caution in carefully chosen populations and with an eye 
to learning as much as possible about how to do it better. 



NOTES 



1 .In this paper we use the term "Internet survey" for both email and HTML form-based 
surveying while the term "Web-based survey" is reserved for HTML form-based 
surveys. 

2. Cook, Heath, & Thompson (2000) included studies of both Web- and email-based 
surveys. 

3. Detailed documentation for the Web Survey Mailer System is provided. However, 
installing and using these tools requires a good working knowledge of HTML and some 
background and understanding of server-based programming. 

4. This Digest is based on an article first appearing in Practical Assessment Research 
and Evaluation 
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